Multinomial Emoji Prediction Using Deep Bidirectional Transformers and Topic Modeling

As social networks grow more prominent, emojis become highly popular with a huge number of users. The primary function of emojis is to fill in emotional cues that might otherwise be missing from textual communication. These days, emojis are considered to be a large part of popular culture around the world. As a consequence, proper use of emojis in text messages will make you more friendly. Recently, emoji prediction becomes a challenging task, due to the large number of classes and the lack of suitable datasets. In this paper, we propose a multimodal emoji prediction model using the contextual information and the visual information. We used the EfficientNetB7 network to extract information from the images. EfficientNetB7 gives us the 10 most likely classes for each image. Also, we used Latent Dirichlet Allocation (LDA) as topic modeling to find hidden topics in the text. The topics extracted by LDA are then combined with the BERT network to improve the performance of this network. On our collected dataset, we achieved 46% accuracy for five emojis and 36% accuracy for 10 emojis.